Project Overview

The February 2024 NVIDIA Generative AI on RTX Contest: https://www.nvidia.com/en-us/ai-data-science/generative-ai/rtx-developer-contest/ was a competition that challenged developers to create a LLM (Large Language Model) that works with and was built by the Tensor-Rt engine to optimize it for speed on inference. I also added some components that let it work on consumer computer gpu and cpu hardware.

Key Elements

• Python Programming
• Large Language Model and Google Collab

Programs Used

• Python Code
• Large Language Model and AI Engines

Approach

I first got the model from github from a researcher's open project for offloading transformer layers and then I modified it with python code and trained it in google colab to work with and be optimized on a Tensor-Rt engine. I also added some safety features (such as using the '//' method to increase the importance of following identity and instructions for conversations).

Results

The AI program is able to generate text in a faster pace on a consumer-grade computer and was trained with Tensor-RT engine on a consumer-grade computer with a quick inference rate. The Tensor-RT engine was not originally designed to work with the Mixtral model so I had to modify it.

Github Repository Link

Google Collab Files: https://github.com/viasky657/GoogleCollabFiles

Project Overview

This project was created as a plugin for Veadotube with Python for personal growth with the goal of allowing an AI to control the Soundboard so that it changes the emotions of the icon of the pngtuber to match the emotion in the user's voice as detected by the LLM (Large Language Model) AI.

Key Elements

• Python Programming
• Generative AI

Programs Used

• Visual Studio Code
• Github Desktop
• Python
• Generative AI

Approach

I got the Roberta Go emotions LLM (Large Language Model), which is a LLM trained on Twitter post datasets, and I used Claude Sonnet 3.5 for some code and I used Python and Visual Studio Code for setting up the environment. I listed all the variables the program will need and how to order them and execute the program in the right order together as a guide.

Results

The plugin was able to free up my attention so I can play games and focus on the chat over having to keep my hands ready to switch emotions on the soundboard for streaming. I learned a lot about coding and Generative AI through this project.

Inquiry about this Project

This project is hosted on my private Github repository, but I can provide you access if you are interested in trying the program yourself if you send a request to my email or social media.